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1 .  Statement  of  the  problem 


Methods  for  an  interactive  extraction  of  roads  from  aerial 
imagery  have  been  developed  and  demonstrated  successfully  /1, 2/. 
This  report  contains  a  description  of  three  major  goals  for  im¬ 
provement  and  extension  of  those  methods  already  existing: 

development  of  an  automatic  procedure  to  identify  and  locate 
starting  points  for  the  extraction  methods 
development  of  several  modifications  to  solve  the  problems 
of  the  extraction  of  line  shaped  objects  other  than  roads 
test  and  evaluation  of  all  methods  in  large  area  imagery 

It  was  the  aim  of  this  research  project  to  develop  and  demon¬ 
strate  the  automatic  extraction  of  certain  classes  of  line  shaped 
objects  from  large  area  imagery  (aerial  images  9"  by  9"  at  a 
scale  of  1  :  7 ^  000).  More  specifically,  we  addressed  the 
problems  of  the  extraction  of  roads,  rivers,  railroads  and  multi¬ 
lane  highways,  where  the  operator  only  has  to  input  certain 
feature  parameters  describing  the  object  class,  the  image  scale, 
etc.  and  the  extraction  starts  and  proceeds  autonomously. 

The  implementation  of  the  different  methods  as  a  single  computer 
program  system,  which  runs  on  our  computer  VAX  11/780,  is  descri¬ 
bed  in  a  seperate  report  /3/.  In  the  following,  a  description 
of  the  methods  will  be  presented  and  the  results  will  be  docu¬ 
mented  and  discussed.  A  complete  printout  of  the  computer  program 
is  added  as  a  separate  document.  A  magnetic  tape  with  all  source 
files  is  available  from  USAETL. 


2.  Short  description  of  two  methods  for  the  extraction  of  line 
shaped  objects  from  aerial  photographs 

To  solve  different  problems  of  the  extraction  of  line  shaped 
objects,  two  different  methods  have  been  developed  which  comple¬ 
ment  each  other  perfectly.  The  basic  idea  of  the  extraction 
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methods  is  the  analysis  of  one  or  more  locally  limited  gray  level 
functions.  As  we  do  not  process  the  whole  image  systematically 
to  produce  locally  isolated  line  segments,  but  proceed  on  an  ob¬ 
ject  guided  basis  from  already  known  line  segments  to  look  for 
a  continuation  of  the  line  object,  we  can  predict  the  most  pro¬ 
bable  location,  orientation  and  properties  of  possible  neigh¬ 
bouring  segments.  Thus  we  even  do  not  have  to  process  2D  sub¬ 
matrices  of  pixels,  but  can  confine  the  computations  to  the 
analysis  of  ID  gray  level  profiles,  which  are  perpendicular  to 
the  predicted  object  orientation.  More  specifically,  we  are 
looking  for  the  very  next  cross  section  or  a  limited  sequence  of 
successive  cross  sections  of  the  object  to  be  extracted. 

In  the  following,  a  short  description  of  the  basic  operator  for 
gray  level  profile  analysis  and  the  two  extraction  methods  is 
presented.  For  more  details  see  /4,5,6,7/. 

2 . 1  Profile  analysis  operator 

The  profile  analysis  operator  (PAO)  performs  a  special  analysis 
of  a  ID  gray  level  function  g  =  f(x).  It  is  the  purpose  of  the 
PAO  to  produce  for  each  point  of  the  function  a  profile  charac¬ 
teristic  pc  which  consists  of  two  profile  element  digits  ped,  and 
ped  ? 


pc  =  ( ped , ,  ped? ) 

Let  v  and  e  be  two  variables  which  may  be  called  gray  level 
variance  and  range  variance.  The  definition  of  the  function  to 
calculate  pc  is  given  indirectly  through  the  two  functions  to 
calculate  ped,  and  ped2  : 


r 


ped , ( x )  = 


3  - 


1 :  i)  g(r)  <  g(x)  +  v  V  r 


*  [  x-e  ,  x)  c  IR 


ii)  3  r0  t  [x-e,  x)  c  R: 
g(r)  <  g(x)  -  v 


V  r  «  [  x-e  ,  rj  c  IR 


2:  i)  g(r)  >  g(x)  -  v  Vri  [x-e,  x)  C  R 

ii)  3  r0  «  [x-e,  x)c  R: 

g(r)  >  g(x)  +  v  V  r  e  [x-e,  rj  c  R 


3:  g(x)-v  <  g(r)<  g(x)+v  Vr  t  [x-e,  x)c  R 


0:  else 


ped? ( x)  -  < 


1:  i)  g(r)  <  g  ( x )  +  v  vr«  (x,  x+e]c(R 

ii)  3  rj  t  (x,  x+e]  C  R: 

g  ( r )  <  g(x)  -  v  V  r  t  [r0  >  x+  e  ]  c  IR 


2:  i)  g  ( r)  >  g(  x )  -  v  V  r  t  (x,  x+e  ]  C  R 

ii)  3r0t  (x,  x+e]cR: 

g(r)  >  g(x)  +  v  Vr  t  [r0  ,  x+e]cR 

3:  g(x)-v<  g(r)  <  g(x)+v  Vr  t  (x,  x+e]cR 


0: 


else 
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For  simplicity  reasons,  the  pair  of  ped,(x)  and  ped?(x)  values 
will  be  written  as  a  number  with  two  digits,  which  is  then  called 
profile  characteristic  pc  of  g  at  the  location  x.  The  set  of 
possible  pc  values  is 

joo,  01,  10,  11,  02,  20,  22,  03,  30,  33,  12,  21,  13,  31,  23,  32} 

Fig.  1  gives  an  example  of  the  calculation  of  pc(x) : 

A  rectangle  with  a  height  of  2  *  v  and  a  length  of  2  *  e. 


Fig.  1:  Example  of  the  calculation  of  pc(x). 

is  centred  on  the  location  (x,  g(x)).  As  the  curve  g(x)  inter¬ 
sects  only  the  bottom  side  of  the  rectangle,  we  get  the  value  1 
for  both  ped,(x)  and  ped2(x)  functions.  Thus  the  profile 
characteristic  at  the  location  x  results  in 

pc( x)  =  1  1 

To  explain  the  meaning  of  the  different  pc's,  it  can  be  stated, 
that  pc(x)  =  11  results  from  a  local  maximum  of  g  at  or  near 
x  and  that  pc(x)  =  22  results  from  a  local  minimum  of  g  at 
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or  near  x,  etc.  Note,  that  the  pc  values  are  computed  by 

considering  a  smaller  or  larger  neighbourhood  of  the  respective 

location  x,  where  this  neighbourhood  is  defined  by  the  variab-  j 

les  v  and  e. 

I 

►t 

The  second  processing  step  of  the  PAO  leads  to  a  partitioning  of 
the  definition  domain  of  g(x)  into  different  intervals:  all 
neighbouring  locations  x  with  identical  pc  constitute  an  interval. 

Each  interval  carries  as  an  index  the  pc  of  the  locations  which 
it  comprises.  In  general,  the  main  property  of  the  PAO  is  that 
it  expands  local  features  of  single  points  such  as  "local 

maximum  in  x"  to  intervals  of  the  definition  domain  which  con-  : 

tain  x  Note,  that  different  partitioning  into  intervals  of  j 

a  function  g(x)  will  result  of  the  application  of  the  PAO  with  I 

different  values  for  the  variables  v  and  e.  For  some  applications 
of  the  PAO  to  be  explained  in  the  following,  only  one  predeter¬ 
mined  set  of  values  for  the  variables  v  and  e  is  used.  For  other 
applications,  several  sets  of  values  for  v  and  e  are  used  sequen¬ 
tially  and  the  stability  or  instability  of  the  resulting  set  of 
partitions  is  evaluated. 


2 • 2  The  local  method  for  line  extraction 

The  local  method  for  extraction  follows  each  line  object  on  a 
step  by  step  basis.  The  basic  procedure  is  the  following: 
Suppose  that  some  parts  of  a  line  object  have  already  been 
extracted.  With  the  knowledge  about  these  object  parts, 

the  location 
the  orientation 

the  necessary  object  features  such  as  contrast  to  the 
surrounding,  width,  pc  of  a  cross  section 
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of  a  possible  continuation  of  the  line  object  are  predicted  and 
tested  via  gray  level  diagram  analysis.  In  the  case  of  compliance 
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with  the  predicted  features,  this  next  increment  of  the  line 
object  is  accepted.  In  the  case  of  noncompliance ,  more  sophis¬ 
ticated  algorithms  are  applied  to  decide,  whether  the  line  object 
continues  with  changed  feature  values  or  ends  at  that  point.  The 
following  main  parts  constitute  the  local  method  to  be  described 
in  more  details. 


Consider  the  case  of  an  idealized  example  shown  in  fig.  2.  The 
sample  points  P;.,  and  P;  of  the  line  object  have  already  been 
extracted.  The  centre  of  the  next  sample  line  to  detect  the 
continuation  of  the  object  is  predicted  on  a  straight  line 
extrapolation  beyond  P; .  The  sample  line  is  a  part  of  a  circle; 
it  extends  symmetrically  on  both  sides  of  the  centre.  The  dis¬ 
tance  between  P;  and  the  centre  is  called  step  width  s  of  the 
method,  which  normally  has  a  default  value  of  some  pixels,  depen¬ 
ding  on  the  scale  of  the  image  and  the  dimensions  of  the  objects. 


a)  prediction  of  point  Pj*$ ; 

b)  gray  level  diagram  along  the  sample  line. 

I 

■V 
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The  pixels  of  the  sample  line  constitute  a  ID  gray  level  diagram 
(see  fig.  2b) ,  which  is  analysed  by  the  PAO  to  verify  a  possible 
continuation  of  the  object.  During  the  extraction  process,  the 
normal  step  width  s  can  be  changed  for  several  reasons: 

if  the  analysis  of  the  respective  gray  level  diagram 
is  negative  or  ambiguous,  the  step  width  is  decreased 
until  a  minimum  value  (usually  1  pixel)  is  reached. 

This  decrease  follows  the  idea,  that  the  gray  level 
diagram  at  a  shorter  distance  from  the  current  stand 
point  can  be  interpreted  more  easily 

if  the  course  of  the  object  deviates  from  a  straight 
line  continuation  or  if  some  object  features  are  changing, 
the  step  width  is  decreased  for  the  following  step 
if  several  extracted  sample  points  of  the  objects  form 
a  straight  line,  the  default  step  width  is  increased  step 
after  step  up  to  a  maximum  value.  This  increase  is 
justified  by  the  fact  that  the  straight  line  object 
continuation  to  be  expected  needs  fewer  sample  points  for 
the  extraction. 


Analysis  of  the  sample  line  gray  level  diagram 

The  analysis  of  a  new  sample  line  gray  level  diagram  is  performed 
by  the  PAO  using  predicted  values  for  v  and  e.  Prediction  is  based 
on  the  v  and  e  values  of  the  last  few  successful  analysis  results 
of  the  PAO.  All  intervals  of  the  partition  of  the  gray  level 
diagram  are  checked  with  respect  to  the  following  features,  the 
values  of  which  are  predicted  from  the  last  few  steps  along  the 
object: 


average  gray  level  of  the  centre  pixels  of  the  interval 
width  of  the  interval 
pc  of  the  interval 
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In  the  ease  of  compliance  with  the  predicted  values,  the  interval 
is  accepted  as  the  cross  section  of  the  object  continuation.  The 
centre  point  of  the  interval  is  added  to  the  list  of  object 
points  and  is  used  as  a  basis  for  the  search  of  the  next  increment 
of  the  object. 

In  the  case  that  the  features  of  more  than  one  interval  of  a 
partition  comply  with  the  predicted  values,  object  splitting, 
noise,  or  other  unfavourable  effects  are  suspected  to  be  present 
at  that  location.  As  no  straight  forward  decision  for  one  of 
the  candidates  is  possible,  a  new  location  for  the  object  conti¬ 
nuation  is  predicted  at  a  shorter  distance  from  the  last  object 
point  and  the  gray  level  diagram  analysis  is  repeated  completely. 
In  the  case  that  no  interval  complies,  a  possible  end  of  the 
object  or  a  gap  in  the  object  is  suspected  at  that  location.  The 
gray  level  analysis  is  repeated  at  a  shorter  step  width. 


Adaption  of  object  features  at  each  step 

After  the  acceptance  of  a  new  object  point,  the  next  step  is 
prepared  by  adapting  the  object  features.  The  following  parameters 
are  adapted  at  each  step: 

PAO  parameters  v  and  e 

object  features:  average  gray  level 

width  of  the  object 
pc  of  the  object 

step  width 


PAO  parameters  and  object  features  are  updated  by 
respective  values  of  the  last  few  steps.  The  step 
on  the  straightness  of  the  last  few  increments  of 
the  degree  of  compliance  of  their  features. 


averaging  the 
width  depends 
the  object  and 
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Extrapolation  steps  to  bridge  small  gaps  of  the  object 

An  extrapolation  step  has  a  bigger  step  width  than  the  maximum 
step  width  under  normal  conditions.  The  extrapolation  step  is 
applied  if,  at  the  current  location,  no  normal  continuation  of 
the  object  was  detected  for  several  gray  level  diagram  analyses 
with  decreasing  step  width  including  the  minimum  step  width. 

As  the  reliability  of  detected  object  continuations  decreases 
with  increasing  step  width,  the  requirements  for  compliance  of 
the  object  feature  values  with  the  predicted  values  are  higher 
than  in  the  normal  case.  For  large  extrapolation  steps  (15  to 
20  pixels) ,  the  emphasis  is  placed  on  the  analysis  of  the  diagrams 
of  two  concentric,  semicircular  sample  lines  where  a  possible 
object  continuation  must  be  detected  on  both  sample  lines. 

Thus  it  is  possible  to  bridge  smaller  gaps  of  the  object,  provided 
that  the  object  continuation  can  be  detected  unambiguously  beyond 
the  gap.  Larger  gaps  or  a  series  of  several  small  gaps  cannot 
and  should  not  be  bridged  by  this  extraction  method  as  there  is 
too  little  evidence  for  accepting  those  results  as  reliable  data 
on  the  basis  of  this  procedure.  The  regional  method  to  be  de¬ 
scribed  in  the  next  section  is  applied  in  these  cases. 


Analysis  of  ob.iect  intersections 

An  intersection  of  line  objects  may  be  present  at  a  location, 
where  the  following  conditions  are  true: 

the  results  of  a  series  of  analyses  of  gray  level 
diagrams  with  decreasing  step  width  contain  two  or 
more  than  two  acceptable  candidates  for  possible 
object  continuations 

the  width  of  a  detected  candidate  is  much  larger  than 
the  expected  value. 


In  these  cases,  the  gray  levels  of  a  full  circle  sample  line 
around  the  suspect  location  are  presented  to  the  PAO.  The  result 
of  this  analysis  is  used  to  confirm  the  intersection  or  discard 
it:  if  only  two  acceptable  candidates  are  detected,  where  one 
of  both  is  located  on  the  object  already  extracted,  the  cue  for 
intersection  is  deleted.  If  more  than  two  acceptable  candidates 
are  detected  with  more  than  one  pointing  in  new  directions  which 
were  not  yet  covered,  an  intersection  is  defined  at  that  location 
All  these  candidates  are  taken  for  possible  object  continuations 
and  are  processed  sequentially. 


2.3  The  regional  method  for  line  extraction 

A  detailed  description  of  this  method  has  already  been  produced 
for  ERO  /I/.  For  the  completeness  of  this  report,  a  summary  of 
the  basic  ideas  is  presented  here. 


The  regional  method  works  in  a  predetermined  limited  part  of  the 
image  (prediction  of  the  area  to  be  processed).  This  area  called 
"area  of  interest"  usually  has  a  rectangular  shape.  The  area  of 
interest  is  expected  to  contain  completely  an  interesting  part 
of  the  line  object  to  be  extracted  (see  fig.  3  a). 


a) 


,  y  X  k 

b)  c) 


Fig.  3:  Characteristics  of  the  regional  method. 


In  addition  the  orientation  of  the  rectangle  is  supposed  to  comply 
with  the  predominant  orientation  of  the  object.  It  has  been  shown 
that  a  complete  analysis  of  the  2D  gray  level  matrix  of  the  area 
of  interest  is  not  necessary  to  detect  the  continuation  of  a  line 
object.  The  analyses  of  several  ID  sample  lines  which  possibly 
contain  cross  sections  of  the  object,  has  proven  to  be  sufficient. 

The  sample  lines  are  perpendicular  to  the  main  axis  of  the 
rectangle.  The  first  part  of  the  line  detection  algorithm  consists 
of  a  gray  level  analysis  of  the  sample  lines  to  discover  candida¬ 
tes  for  object  points.  A  confidence  value  is  computed  for  each 
point  of  each  sample  line  to  indicate  its  probability  of  belonging 
to  a  line  object.  A  semi  threshold  operation  is  applied  to  this 
matrix  of  confidence  values  to  retain  only  a  few  points  with  the 
highest  values  of  each  sample  line  (see  fig.  3  b) . 

The  second  part  of  the  algorithm  works  on  this  set  of  candidates 
to  detect  subsets,  the  points  of  which  are  in  a  collinear 
position.  This  computation  is  done  iteratively  to  increase  the 
confidence  value  of  a  point,  if  collinear  continuation  points 
exist,  or  to  decrease  its  confidence  value  otherwise.  Thus  it 
is  possible  to  confirm  all  candidates,  which  contribute  to  form 
a  line,  while  the  more  or  less  isolated  candidates  can  be  dis¬ 
carded.  At  the  same  time,  two  vectors  have  been  computed  for  each 
confirmed  candidate,  which  point  to  both  neighbouring  candidates 
in  a  line  above  and  below.  On  the  basis  of  this  direction  infor¬ 
mation  and  the  last  known  object  point  P,  the  line  following  task 
is  accomplished  as  the  last  part  of  this  algorithm  (see  fig.  3  c). 

Attention  should  be  given  to  the  fact,  that  the  local  and  the 
regional  method  for  line  object  extraction  complement  each  other 
very  well:  While  the  first  method  is  able  to  follow  any  curvature 
of  the  line  object,  to  adapt  changing  features,  to  accommodate 
the  necessary  amount  of  computation  and  to  detect  line  intersec¬ 
tions,  the  second  method  is  able  to  process  noisy  image  data,  to 
bridge  gaps  and  to  cope  with  badly  visible  objects.  Thus  the 
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cooperation  between  both  methods  in  combination  with  a  method 
for  the  automatic  definition  of  starting  points  on  the  line 
objects  provides  a  maximum  of  correctness  and  reliability  of 
object  extraction  results  with  a  minimum  of  computation. 


3.  Description  of  a  method  for  the  recognition  of  starting 
points  for  the  automatic  extraction  of  roads 

The  extraction  methods  work  on  an  object  guided  basis,  i.e.  from 
the  position  of  the  lastly  detected  object  segment,  the  position 
of  a  continuing  object  segment  is  predicted  and  the  computations 
concentrate  on  that  predicted  position  to  verify  or  reject  a  new 
object  segment.  This  basic  procedure  requires,  that  at  least  one 
initial  starting  position  for  each  seperate  object  is  given.  As 
a  first  and  most  obvious  method  to  define  these  starting  points, 
an  interactive  procedure  has  been  implemented,  where  the  human 
interpreter  detects  a  suitable  starting  point  visually  and  inputs 
the  coordinates  manually.  An  automatic  method  for  this  task  has 
also  been  developed  and  implemented,  which  works  as  follows: 
Several  lines  and  columns  of  the  image  matrix  are  selected  as 
sample  lines  to  be  analysed  by  the  PAO.  The  values  of  the  varia¬ 
bles  v  and  e  are  derived  from  the  type  of  the  object  and  the 
resolution  of  the  image.  All  sample  lines  are  partitioned  into 
intervals  as  described  in  section  2.1.  Only  those  intervals  are 
accepted  as  candidates  for  starting  points,  the  features  of 
which  comply  with  the  predetermined  object  features,  e.g.  if  we 
are  interested  in  extracting  roads  of  a  certain  width  w  and  a 
positive  contrast  with  the  surroundings,  only  the  intervals  with 
pc  =  1 1  and  width  w  are  accepted  as  candidates. 
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For  verification,  each  candidate  is  analysed  more  closely  by  the 
following  procedure  (see  fig.  4). 


Fig.  4;  To  explain  the  verification  of  a  starting  point. 

Two  concentric  circular  sample  lines  around  the  candidate  location 
are  submitted  to  the  PAO  for  gray  level  analysis.  The  interpreta¬ 
tion  of  the  partitioning  of  the  sample  lines  has  to  comply  with 
several  requirements.  A  candidate  is  discarded  as  soon  as  the 
first  noncompliance  occurs: 

there  must  be  at  least  two  intervals  on  the  smaller 
circle,  which  are  located  diametrically  and  the  features 
of  which  (average  gray  level,  width,  pc  value)  comply 
with  the  respective  features  of  the  candidate 

there  must  be  exactly  two  intervals  on  the  bigger  circle 
such  that  they  are  collinear  with  a  pair  of  diametrical 
intervals  of  the  smaller  circle  and  that  the  features  again 
comply  with  the  respective  features  of  the  candidate 
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the  gray  level  variance  along  a  sample  line,  which  fits 
through  the  centres  of  all  selected  intervals  (i.e.  this 
sample  line  coincides  with  the  suspected  line  object  at 
that  region,  see  fig.  4)  is  lower  than  a  certain  threshold 
value,  the  amount  of  which  is  set  as  a  default  value  or 
can  be  chosen  interactively. 

If  these  requirements  are  fulfilled,  we  accept  the  candidate  as 
a  true  starting  point  for  an  extraction  attempt  and  -  as  the  re¬ 
quirements  are  quite  stringent  -  we  are  certain  that  it  is  a 
reliable  starting  point  in  most  cases.  Test  results  of  this 
method  are  shown  in  section  5. 

4.  Modifications  for  the  extraction  of  rivers  and  highways 
4.  i  MQdifi.ca.ti.Qria  for..  riv-era. 

It  has  been  found  that  no  major  modifications  of  the  methods  were 
necessary  to  handle  the  problems  arising  with  the  extraction  of 
rivers  from  images,  as  long  as  they  comply  with  the  requirement 
of  a  line  object:  extreme  length/width  relation;  width  not  bigger 
than  approx.  10  pixels  in  an  image  matrix  (the  latter  requirement 
can  be  satisfied  for  a  wide  range  of  applications  by  variation 
of  image  scale  and  digitization  resolution). 

The  necessary  modifications  refer  to  some  feature  values  only. 

In  normal  aerial  images,  rivers  appear  with  negativ  contrast  to 
the  surroundings,  which  is  taken  into  account  by  defining 
pc  =  22  for  all  methods.  At  the  same  time,  the  width  of  rivers 
usually  varies  to  a  higher  extend  than  the  width  of  roads,  which 
can  be  accommodated  by  a  higher  threshold  for  the  width  variation 
tolerance  in  all  methods.  Low  contrast  with  the  surroundings  or 
high  curvature  require  the  frequent  change  between  both  extraction 
methods  while  operating  in  a  cooperative  mode.  Results  are  shown 
in  section  5. 


The  application  of  the  method  for  recognition  of  starting  points 
(see  section  3)  to  the  problem  of  multi-line  objects  revealed 
two  substantial  constraints: 


the  larger  width  of  these  objects  could  not  be  handled 
adequately 

The  compound  profile  of  such  an  object  could  not  be 
handled  adequately  while  analysing  the  gray  level 
diagram  of  cross  section  sample  lines. 

To  solve  these  problems,  the  method  for  recognition  of  starting 
points  has  been  modified  to  process  the  image  data  at  a  reduced 
resolution  and  completed  by  a  special  verification  part.  The  basic 
procedure  now  follows  these  ideas: 

In  comparison  with  the  processing  of  normal  single  line  objects, 
we  reduce  the  resolution  of  the  image  by  a  factor  of  3  to  5.  After 
this  operation,  the  separate  lines  of  most  multi-line  objects 
merge  into  a  single  line,  which  -  as  the  first  part  of  the 
starting  point  recognition  -  allows  the  application  of  the  origi¬ 
nal  method  with  suitable  parameter  values.  This  reduction  of 
resolution  causes  a  substantial  loss  of  object  details  which  leads 
to  a  higher  rate  of  misclassif ications  and  false  alarms,  as  a 
lot  of  objects  resemble  each  other  at  that  resolution  (fields, 
yards,  treeless  areas  in  a  forest,  etc.).  To  compensate  this  de¬ 
ficiency,  a  verification  has  been  added  to  decide,  whether  a  pre¬ 
liminary  starting  point  which  has  been  accepted  at  the  reduced 
resolution,  can  be  accepted  as  a  true  starting  point  for  a  multi- 
line  object.  This  verification  algorithm  works  in  the  original, 
non-reduced  image  data  resolution  and  essentially  applies  the 
same  basic  computations  as  described  above.  The  preliminary 
starting  point  location  is  transformed  from  the  reduced  resolution 
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Fig .  5 :  Details  of  the  verification  of  a  preliminary  starting 
point  for  a  highway 

a)  gray  level  profile;  b)  gray  levels  of  the  sample 
lines  of  an  area  of  interest;  c)  the  resulting  pc  values 
of  the  PA0. 
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image  back  into  the  non-reduced  resolution  image.  An  area  of 
interest  is  placed  at  this  location  and  several  sample  lines  are 
submitted  to  the  PAO  for  gray  level  analysis.  It  is  the  aim  of 
this  analysis  of  gray  level  diagrams  perpendicular  to  the 
expected  orientation  of  the  object  in  the  full  resolution  image 
to  detect  the  individual  parallel  lines  (e.g.  the  different  lanes 
of  a  highway)  for  a  comparison  with  the  expected  number,  width 
and  sequence  (see  fig.  5).  An  average  width  is  computed  for  the 
intervals  on  all  gray  level  diagrams  which  possibly  represent 
the  same  line  of  the  object.  Explaining  the  example  of  fig.  5 
further,  we  get  a  sequence  of  one  bright,  one  dark,  and  one 
bright  line  with  an  average  width  of  4  pixels,  1  pixel,  and  4 
pixels  respectively.  If  we  searched  for  a  starting  point  on  a 
highway  of  this  type  and  dimensions,  we  would  finally  accept 
this  location  as  a  true  starting  point  and  transfer  the 
coordinates  to  the  extraction  method. 


Only  the  regional  method  has  been  modified  appropriately  to 
accommodate  the  problems  of  extraction  of  multi-lane  highways. 

The  modifications  refer  to  a  reduction  of  the  image  resolution 
for  the  normal  extraction  steps  with  the  regional  method  and  some 
verification  tests  which  are  performed  in  the  original  high  reso¬ 
lution  image  to  assure  the  reliability  of  the  results  and  to 
exclude  deviations  of  the  extraction  process. 


Due  to  the  processing  of  the  reduced  resolution  image  the 
extraction  speed  profits  from  a  significant  reduction  of  the 
amount  of  image  data;  on  the  other  hand  we  need  not  worry  about 
3harp  bends  which  would  complicate  the  extraction  by  the  regional 
method:  at  a  reduced  resolution  image  the  highways  mostly  follow 
a  straight  line  and  have  only  smooth  curves. 


After  a  predetermined  number  of  extraction  steps,  the  verifi¬ 
cation  test  as  described  in  section  4.2  is  performed  to  assure 
that  the  current  multi-line  object  is  still  in  the  focus  of  the 
extraction  process  and  no  deviation  to  follow  exits,  neighbouring 
fields  or  tree  lines  has  occurred.  If  the  verification  test 
reveals  any  doubt  about  the  correctness  of  the  extraction  process 
(e.g.  no  compliance  of  the  sequence  of  the  different  lines  with 
the  expected  sequence) ,  the  verification  is  repeated  immediately 
after  the  next  extraction  step.  If  the  doubts  persist,  the 
extraction  is  stopped  and  reinitialized  at  the  next  starting 
point . 

Thus  we  succeed  again  to  produce  reliable  extraction  results  with 
a  high  degree  of  automation,  taking  into  account  the  minor  disad¬ 
vantage  that  we  sometimes  miss  a  noisy  or  distorted  part  of  the 
object.  But  the  interactive  completion  of  missing  object  parts 
is  performed  much  more  easily  than  the  correction  of  false 
results  . 
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5.  Results 

5. 1  Results  of  automatic  extraction  of  roads  from  small  scale. 
Large  area  aerial  photographs 

The  method  for  recognition  of  starting  points  and  the  two  methods 
for  extraction  of  roads  -  as  described  in  sections  2  and  3  of 
this  report  -  have  been  implemented  and  tested  separately  on  our 
computer  DEC  PDP  11/70.  After  having  achieved  satisfactory 
results,  all  three  methods  were  implemented  for  cooperative  road 
extraction  as  a  single  computer  software  system  on  our  DEC  VAX 
11/780.  For  details  of  the  implementation  refer  to  the 
supplementary  report  / 3/. 

Test  results  are  shown  and  explained  for  an  aerial  photograph  of 
20  cm  x  20  cm  at  a  scale  of  1  :  74  000  as  supplied  by  USAETL. 

This  image  has  been  digitized  on  an  OPTRONICS  scanner  to  produce 
an  image  matrix  of  8196  x  8196  pixels  with  8  bit  gray  level  reso¬ 
lution.  The  image  matrix  has  been  partitioned  into  a  mosaic  of 
17  x  17  overlapping  subimages  of  512  x  512  pixels  each.  Each 
subimage  corresponds  to  an  area  of  approx.  1  km  x  1  km  and  needs 
-  as  an  average  value  -  a  computation  time  of  approx.  1  min, 
input  data  transfer,  processing  and  storing  of  the  results  inclu¬ 
ded.  Fig.  6  shows  the  test  image  with  an  overlay  to  indicate  the 
location  of  each  of  the  289  subimages. 

A  short  analysis  of  the  image  contents  revealed  that  a  great 
variety  of  roads  is  present  in  the  imaged  area.  To  get  a  complete 
overview  of  the  methods 's  performance,  we  processed  this  image 
several  times  with  different  values  for  the  parameter  "width". 

All  other  parameters  were  set  to  their  default  initial  values 
and  were  adapted  during  the  process  as  described  in  section  2. 

The  results  of  the  different  runs  are  presented  in  the  following 
figures . 
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The  first  run  was  started  with  an  object  width  tolerance  of  2 
to  4  pixels.  This  means  that  we  address  the  small  roads  or  sepa¬ 
rate  single  lanes  of  highways  selectively,  because  for  our  digi¬ 
tization,  a  pixel  corresponds  to  a  circular  area  of  approx.  2  m 
diameter  on  the  ground.  Fig.  7  shows  the  result  of  this  run, 
which  needed  4  :  43  hours  of  processor  time  on  the  VAX  11/780 
for  the  complete  mosaic  of  289  subimages,  including  data  transfer 
to  and  from  the  disc  storage. 

It  can  be  seen  that  a  lot  of  connected  roads  has  been  extracted, 
especially  in  the  left  half  of  the  image.  Furthermore,  there 
appear  some  short  segments  of  roads  isolated  from  each  other 
(note,  that  at  this  scale,  a  line  segment  of  1  cm  corresponds 
to  a  road  length  of  approx.  700  m  on  the  ground)  and  there  remain 
some  gaps  between  large  connected  parts  of  the  network  of  roads. 

A  second  run  was  initialized  with  an  object  width  tolerance  of 

3  to  6  pixels  (corresponding  to  approx.  6  m  to  12  m  on  the  ground 

to  address  the  medium  size  roads.  The  result  is  shown  in  fig.  8. 

The  processing  time  for  this  run  was  4  :  49  hours  in  total. 

With  reference  to  fig.  6  and  fig.  7,  it  can  be  seen  that  most 

of  the  small  roads  were  also  detected  during  this  run  that  some 
gaps  in  the  results  of  the  first  run  could  be  bridged  (see  sub¬ 
images  No.  58,  59,  156,  193,  215,  etc.)  and  that  a  considerable 
amount  of  extended  parts  of  roads  have  been  detected  in  addition 
to  the  results  of  the  first  run  (especially  in  the  right  half 
of  the  image,  see  subimages  No.  28,  44,  61,  78,  95,  112,  113, 

233,  234,  218,  219,  etc.). 


Extraction  result  for  medium  size  roads  of  3  to  6  pixels. 
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A  third  run  was  initialized  to  address  the  broad  roads  and  high¬ 
ways.  The  width  tolerance  was  set  to  5  and  8  pixels.  The  results 
are  shown  in  fig.  9. 

The  processing  time  of  this  run  amounted  to  3  :  24  hours.  It  can 
be  seen  from  the  results,  that  practically  none  of  the  smaller 
roads  is  detected  in  this  run,  but  a  few  parts  of  the  network 
of  roads,  especially  in  the  lower  right  section  of  the  image, 
have  been  found  in  addition  to  the  previous  results.  The  process¬ 
ing  time  is  only  by  a  factor  of  1/3  shorter  than  for  the  first 
two  runs;  this  is  due  to  the  fact,  that  all  subimages  have  to  be 
transferred  to  the  working  storage,  which  takes  a  considerable 
amount  of  the  computing  time,  and  the  search  for  starting  points 
is  performed  in  all  subimages,  but  was  successful  with  this 
parameter  setting  only  in  a  few  subimages. 

As  an  interesting  experiment,  we  superposed  the  results  of  all 
three  runs  to  one  another  (see  fig.  10). 

By  comparison  of  this  results  with  the  aerial  image  (fig.  6)  it 
can  be  stated  that 

the  bigger  part  of  the  network  of  roads  (approx.  80  %  to 
90  %)  has  been  extracted  automatically  with  no  interaction 

practically  no  errors  occurred  during  the  automatic 
extraction  process.  A  very  few  amount  of  false  results 
may  be  noticed  (e.g.  in  subimages  No.  121,  248  and  252) 

a  considerable  number  of  gaps  between  bigger  parts  of  the 
network  of  roads  still  remain.  A  closer  look  at  the  image 
data  reveals  that  at  the  position  of  most  of  these  gaps 
the  respective  road  cannot  be  detected  locally.  In  most 
cases  the  road  is  hidden  by  trees  or  shows  no  contrast  to 
the  surrounding  area.  Some  of  these  problems  will  be 
analysed  in  more  detail  in  the  following. 


-  27  - 

a  few  very  short  segments  of  line  objects  were  detected 
with  no  continuation  (see  subimages  No.  14,  30,  89,  100, 
119,  197).  In  fact,  most  of  these  segments  are  located  in 
forest  areas,  were  starting  points  could  be  detected,  but 
the  extraction  methods  decided  that  no  reliable  continua¬ 
tion  was  possible  (see  subimages  No.  14,  30,  89,  119,  197 
in  fig.  6).  It  would  have  been  an  easy  task  to  erase  these 
spurious  results  on  the  basis  of  a  minimum  length  thres¬ 
hold.  But  we  decided  not  to  do  the  deletion  at  first  hand, 
because  some  of  these  short  segments  are  true  parts  of  the 
roads  and  could  be  very  useful  to  a  later  higher  sophisti¬ 
cated  automatic  method  for  the  completion  of  the  extraction 
process  (see  subimages  No.  33,  53,  133,  150,  175).  These 
short  segments  also  could  support  an  interactive  completion 
of  the  extraction  results,  after  which  an  automatic  post¬ 
processing  would  erase  the  rest  of  the  segments,  which  at 
that  status  of  the  processing  would  have  been  definitely 
identified  as  false  results. 

To  investigate  the  properties  of  the  methods  further,  we  initi¬ 
ated  another  two  runs  with  different  parameter  values.  Fig.  11 
shows  the  results  of  a  run  where  the  object  width  tolerance  was 
set  between  2  and  5  pixels. 

It  turns  out  that  this  run  produced  the  most  complete  result  of 
the  roads,  but  needed  the  highest  amount  of  computer  time 
(5  :  10  hours  for  the  whole  set  of  289  subimages).  The  comparison 
of  this  result  with  the  results  in  fig.  7  and  fig.  8  shows  that 
a  single  run  with  optimized  parameter  values  can  produce  approxi¬ 
mately  the  same  results  as  two  runs  with  a  selective  parameter 
setting  for  specific  object  classes.  This  demonstrates  very 
clearly  both  the  flexibility  and  the  sensitivity  of  the  automatic 
extraction  process,  which  can  be  adapted  to  a  variety  of  extrac¬ 
tion  problems  by  specific  parameter  values. 


Extraction  results  with  a  width  tolerance  of 
4  to  7  pixels. 
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The  next  results  shown  in  fig.  12,  were  produced  by  a  run  with 
an  object  width  tolerance  of  4  to  7  pixels.  But  this  run  was  based 
on  the  results  of  the  previous  run  (see  fig.  11),  so  that  we  only 
extracted  roads  at  locations,  where  the  previous  run  had  produced 
no  result.  We  realized  this  proceeding  by  using  the  result  matrix 
of  each  subimage  (for  details  of  the  result  format  see  /3/)  to 
control  the  method  for  starting  point  recognition  as  well  as  the 
methods  for  object  extraction.  In  contrast  to  this  proceeding, 
the  results  presented  in  fig.  7,  8  and  9  had  been  produced 
completely  independently,  i.e.  no  result  of  one  run  had  been  used 
to  control  the  process  of  the  next  run  or  to  avoid  certain  areas, 
which  had  already  been  processed  by  the  previous  run. 

If  we  do  realize  the  control  of  the  second  run  by  the  results 
of  the  previous  run,  as  shown  in  fig.  11  and  12,  we  get  only  the 
complementary  results  with  the  benefit  of  a  shorter  computing 
time,  which  amounted  to  2  :  54  hours  for  the  complete  mosaic  of 
289  subimages. 

The  superposition  of  the  results  of  these  two  complementary  runs 
is  shown  in  fig.  13.  This  final  result  is  comparable  to  the  super¬ 
position  of  three  runs  (fig.  10)  regarding  the  number  of  errors, 
the  completeness,  the  amount  of  gaps  and  short  object  segments. 

A  detailed  analysis  and  assessment  of  the  extraction  results  of 
some  subimages  will  be  presented  in  the  following  figures.  As 
an  overview,  the  respective  subimages  are  indicated  in  fig.  14: 
subimages  marked  by  %  will  be  shown  -  together  with  the  respective 
results  -  in  a  magnified  representation;  results  of  subimages 
marked  by  §  will  be  explained  on  the  basis  of  line  printer  repre¬ 
sentation.  All  magnified  or  detailed  presentations  show  sections 
of  fig.  10  (superposition  of  the  results  of  the  first  three  runs). 
Reference  is  also  made  to  the  total  image  (Fig.  6). 
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Fig.  14:  Labeling  of  the  subimages 


Fig.  15  shows  the  situation  of  subimage  No.  53*  The  extraction 
result  is  incomplete  due  to  two  gaps  in  the  upper  right  part  of 
the  subimage,  where  the  road  obviously  is  hidden  by  trees.  There 
is  also  a  short  incorrect  segment  of  the  result  in  the  lower  part 
of  the  subimage,  where  the  road  is  not  visible. 


Fig.  15:  Details  of  subimage  No.  53- 


Fig.  16  shows  details  of  subiraage  No.  146.  The  gaps  of  the  result¬ 
ing  road  extraction  are  due  to  extremely  low  contrast  between  the 
object  and  the  surrounding  fields.  The  missing  horizontal  part 
at  the  upper  end  has  been  detected  as  a  continuation  from  subimage 
No.  145  and  is  contained  in  the  results  of  subiraage  No.  129  (refer 
to  fig.  10). 
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Fig.  16:  Details  of  subimage  No.  146. 


Fig.  17  shows  details  of  subimage  No.  150.  In  the  upper  left  part 
of  the  subimage,  the  road  is  not  detectable  due  to  very  low  con¬ 
trast.  In  the  lower  left  part  of  the  subimage,  the  road  is  bor¬ 
dered  by  a  strip  of  approx,  equal  width  and  brightness  (probably 
strips  of  cut  trees  at  the  sides  of  the  road)  which  alltogether 
constitute  a  line  object  much  wider  than  a  normal  road.  Thus  this 
part  of  a  road  could  not  be  detected  even  with  object  width 
tolerances  up  to  8  pixels. 

Fig.  18  shows  details  of  subimage  No.  158.  The  continuation  of 
the  horizontal  segment  of  the  road  in  the  middle  of  the  subimage 
is  invisible  due  to  tree  coverage.  The  vertical  segment  of  the 
road  in  the  right  part  of  the  subimage  should  have  been  extrac¬ 
table  during  the  run  with  the  bigger  width  tolerance,  but  no 
starting  point  seems  to  have  been  detected  on  that  segment.  This 
assessment  leads  to  another  suggestion  for  improvement  of  the 
overall  performance:  if  roads  of  different  width  are  to  be 
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Fig.  17:  Details  of  subimage  No.  150. 


extracted  by  a  sequence  of  runs  with  different  width  tolerance 
values,  don't  search  for  starting  points  during  a  successive  run 
before  the  dead  ends  of  the  previous  run  have  been  used  as 
starting  points? 


i  Details  of  subimage  No.  158. 
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Fig.  19  shows  details  of  subimage  No.  163-  There  is  one  gap  in 
the  vertical  segment  of  the  road,  which  is  totally  invisible  and 
hence  not  extractable.  The  majority  of  the  road  network  has  been 
extracted  correctly  and  completely. 


Details  of  subimage  No.  163- 


Fig.  20  shows  details  of  subimage  No.  210.  The  gap  in  the  centre 
of  this  subimage  result  is  obviously  due  to  presence  of  a  farm 
yard  which  does  not  resemble  to  a  road.  The  other  missing  parts 
of  the  roads  in  the  upper  right  of  the  subimage  could  not  be 
detected  due  to  the  intensive  coverage  by  trees. 
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Fig.  20:  Details  of  subimage  No.  210. 


We  used  this  section  of  the  aerial  image  in  earlier  tests  and 
had  success  with  the  extraction  of  these  parts  of  the  roads  (see 
fig.  37  to  50  in  /I/).  The  success  of  those  test  results  is  due 
to  the  interactively  chosen  area  of  interest  for  the  regional 
method,  whereas  the  results  of  fig.  20  were  produced  automati¬ 
cally.  This  is  another  proof  of  the  superior  control  of  pattern 
recognition  methods  which  can  be  executed  by  a  human  operator. 

Fig.  21  shows  details  of  subimage  No.  215.  There  is  only  one 
vertical  segment  of  the  road  missing  in  the  upper  right  corner 
of  the  subimage  which  is  due  to  the  very  low  or  even  disappearing 
contrast  with  the  fields.  The  majority  of  the  road  network  has 
been  detected  completely  and  correctly. 
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Fig.  21 :  Details  of  subimage  No.  215. 


Fig.  22  shows  details  of  subimage  No.  243.  The  only  small  gap, 
which  could  be  filled  by  simple  extrapolation,  is  obviously  due 
to  the  farm  yard  which  caused  the  extraction  methods  to  stop. 
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Another  product  to  assess  the  performance  of  the  methods  in  detail 
can  be  presented  by  line  printer  outputs  of  the  result  matrix. 

Here  we  can  see  the  number  and  location  of  starting  points  (marked 
by  the  character  "S" )  and  can  assess  the  cooperation  between  both 
extraction  methods,  because  the  results  produced  by  the  local 
method  are  marked  by  the  character  ,  whereas  the  results 
produced  by  the  regional  method  are  marked  by  the  character  "o”. 

Fig.  23  shows  a  line  printer  output  of  the  results  of  subimage 
No.  86.  Only  two  starting  points  were  used  for  the  road  extraction 
in  this  subimage  and  most  of  the  results  were  produced  by  the 
local  method  due  to  the  undistorted  visibility. 

Fig.  24  shows  the  line  printer  output  of  the  results  of  subimage 
No.  210  (see  also  Fig.  20).  It  can  be  seen  from  this  presentation 
that  three  starting  points  were  used  for  the  extraction  of  the 
two  disconnected  parts  of  the  roads,  that  both  methods  had  an 
equal  share  at  the  production  of  the  results  and  that  the  local 
method  decided  to  stop  the  extraction  at  the  farm  yard  in  the 
middle  of  this  subimage. 
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Fig.  24:  Line  printer  output  of  the  results  of  subiraage  No.  210. 
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Only  a  few  test  images  contained  rivers.  After  some  preliminary 
tests,  a  set  of  default  values  for  the  parameters  could  be  derived 
with  one  exception:  the  width  of  rivers  varies  substantially  and 
cannot  be  predetermined  without  knowledge  about  the  scale  of  the 
image  and  the  specific  river  to  be  extracted.  Fig.  25  shows  two 
test  results  of  the  method  for  the  recognition  of  starting  points 
on  rivers.  The  short  white  strokes  at  the  upper  and  lower  edge 
of  the  image  indicate  the  sample  columns,  which  have  been  checked 
for  starting  points.  For  the  first  example,  three  starting  points 
have  been  recognized,  indicated  by  white  strokes  on  the  rivers; 
for  the  second  example,  which  shows  better  contrasts  and  less 
variation  of  object  properties,  six  starting  points  have  been 
recognized,  while  only  one  would  have  been  sufficient  for  the 
latter  extraction  process  in  both  cases. 


a)  b) 

Fig.  25:  Two  examples  for  the  recognition  of  starting  points 
of  rivers. 
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The  extraction  of  the  river  in  fig.  25  a  is  shown  in  the  following 
figure  26.  The  extraction  result  was  produced  by  the  cooperation 


a)  b)  c) 

Fig.  26:  Extraction  of  the  river  shown  in  Fig.  25  a. 

of  the  local  and  the  regional  method.  Only  the  uppermost  starting 
point  has  been  necessary.  The  local  method  starts  at  that  point 
to  extract  the  river  to  the  right  until  the  edge  of  the  image 
is  reached  and  continues  to  the  left  of  the  starting  point,  until 
a  distortion  (see  arrow  in  fig.  26  a)  is  reached.  From  this  point 
the  regional  method  pursues  the  extraction  until  a  curve  is 
reached  (see  arrow  in  Fig.  26  b).  At  that  point  the  local  method 
resumes  the  task  until  more  distortions  cause  another  call  for 
the  regional  method  which  extracts  the  object  down  to  the  lower 
left  corner  of  the  image  (see  fig.  26  c). 
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5.3  Results  of  the  extraction  of  highways  from  aerial 
photographs 

Two  examples  are  presented  to  document  the  results  of  the  extrac¬ 
tion  of  highways.  Fig.  27  shows  the  results  of  the  recognition  of 
starting  points.  Only  very  few  sample  lines  respectively  sample 
columns  were  analysed  at  a  1  :  4  reduced  resolution  of  both 
images.  Four  preliminary  starting  points  were  detected  in  each 
image,  two  of  which  were  accepted  as  true  starting  points  by  the 
verification  algorithm,  which  processed  the  images  at  a  resolution 
as  shown  in  fig.  27.  The  respective  locations  are  marked  by  white 
strokes  superimposed  to  the  image. 


a)  b) 

Fig.  27:  Results  of  the  recognition  of  starting  points  for 
highways . 

Based  on  one  starting  point  for  each  image,  the  extraction  method 
produces  the  results  shown  in  fig.  28.  As  the  highways  are  mostly 
straight  lined  and  contrast  well  with  their  surroundings,  there 
are  no  major  problems  for  the  extraction.  The  highway  exits  in 
both  examples  were  not  detected,  as  the  regional  extraction  method 
in  its  present  implementation  is  not  able  to  handle  this  task. 


d)  e)  f) 

Fie.  28:  Results  of  the  extraction  of  highways. 

6.  Conclusions 

It  has  been  shown  that  the  automatic  extraction  of  a  network  of 
line  objects  from  aerial  images  can  be  performed  reliably  and 
precisely.  The  success  in  generating  correct  extraction  results 
is  sometimes  balanced  by  the  incompleteness  of  the  results,  i.e. 
some  parts  of  the  objects  which  are  occluded,  distorted,  or  noisy, 
will  remain  undetected  on  the  basis  of  these  more  or  less  local 
gray  level  analysis  methods.  To  solve  these  problems  automatically, 


we  need  to  develop  methods  of  a  higher  degree  of  sophistication, 
which  must  be  based  on  the  evaluation  of  context  of  the  near  and 
far  neighbourhood.  Context,  in  this  case,  must  be  understood  as 
preliminary  results,  object  properties,  possible  or  suspected 
object  relations,  implications  of  shadowing,  etc. 

Another  problem,  which  remains  unsolved  as  yet,  is  the  extraction 
of  line  objects  from  densely  populated  areas,  where  a  compound 
of  regularly  shaped,  man  made  objects  complicates  the  local  gray 
level  analysis  to  an  extend,  that  the  success  of  these  methods 
is  considerably  lower  than  in  rural  areas.  Again,  to  solve  the 
extraction  task  in  urban  areas  with  a  comparable  success  rate 
as  shown  above,  we  need  methods  with  a  higher  degree  of  sophisti¬ 
cation  which  must  incorporate  several  operators,  context  evalua¬ 
tion,  iterative  improvement  of  preliminary  results,  knowledge 
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